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NONMEASURABLE IDENTICALLY DISTRIBUTED 
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Abstract. Let SI be a countable infinite product S7 N of copies of the 
same probability space Qi, and let {E n } be the sequence of the coor- 
dinate projection functions from Q, to fii. Let $ be a possibly non- 
measurable function from fii to R, and let X n {ui) = ^(H n (tj)). Then 
we can think of {X n } as a sequence of independent but possibly non- 
measurable random variables on fl. Let S n = Xi + • • • + X n . By 
the ordinary Strong Law of Large Numbers, we almost surely have 
E*[Xi] < liminf S n /n < lim sup S„/n < E*[Xi], where E» and E* 
are the lower and upper expectations. We ask if anything more precise 
can be said about the limit points of S„/n in the non-trivial case where 
i?t[Xi] < and obtain several negative answers. For instance, the 

set of points of £1 where S n /n converges is maximally nonmeasurable: 
it has inner measure zero and outer measure one. 



1. Introduction 

Ordinary random variables are P-measurable functions on a probability 
space (fi, J 7 , P), where J 7 is a a- field on f2. By the ordinary Strong and Weak 
Laws of Large Numbers (LLNs), if X\,X2, ... are measurable identically dis- 
tributed random variables with finite expectation, then (X\ + - ■ • + X n )/n — > 
i?[A"i] almost surely (Strong Law) and in probability (Weak Law). But we 
can also ask what happens to long-term means of samples when the random 
variables are not measurable. 

Let (0, P) be a probability space. The following collects some known 
facts (see, e.g., [3 Lemmas 1.2.2 and 1.2.3]) that allow us to apply proba- 
bilistic techniques in the case of nonmeasurable random variables. 

Proposition 1. Let H be any subset of£l. Then there are measurable sets 
and H* such that C H C H* and such that for any measurable 
A C H we have P{A) < P(H*) and for any measurable B ^ H we have 
P(B) > P(H*). The sets and H* are uniquely defined up to sets of 
measure zero. 

For any real-valued function f on S7 ; there are measurable functions /* 
and f* such that /*</</* everywhere and for any measurable g on f2 
such that g < f everywhere, we have g < /* almost surely, while for any 
measurable h on Q such that f < h everywhere, we have h > f* almost 
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surely. The functions /* and f* are uniquely defined up to almost sure 
equality. 

The functions /* and /* are the maximal measurable minorant and min- 
imal measurable majorant of /, respectively. 

We then have P*(P) = P(P*) and P*(H) = P(H*), where P* and P* are 
the inner and outer measures generated by P. Note that H is measurable 
with respect to the completion of P if and only if P*(H) = P*(H), in which 
case it has that value as its measure with respect to the completion of P. 
If, as some confirmation theorists are wont to do (e.g., [3]), we were to deal 
in interval-valued probabilities, we might reasonably define the probability 
of H as [P*(P),P*(PQ]. 

We say that a set is maximally nonmeasurable provided that P*(H) = 
and P*(H) = 1. Such a set is one all of whose measurable subsets have null 
measure and all of whose measurable supersets have full measure. 

As a replacement for the independence assumption in the case of ordi- 
nary random variables, take our probability space (Q, P, P) to be a prod- 
uct of the probability spaces (Q n ,P n , P n ), and let our sequence of possi- 
bly nonmeasurable random variables be a sequence of functions X±, X2, ■■■ 
on Q such that X n (ui, 002, •••) depends only on the value of u n , so that 
there is a function ty n such that X n (ui,U2, ...) = ^> n (u) n ). We will say that 
X\, X2, ... is then a sequence of independent identically-distributed possibly- 
nonmeasurable random variables (iidpnmrvs) providing that all the proba- 
bility spaces (f2 n , P n , P n ) are the same space (Oi, Pi, Pi) and that *$> n is the 
same function ^1 for all n. 

The following fact about product measures and the (•)* and (•)* operators 
follows from Lemma 1.2.5]. 

Proposition 2. Suppose (f2,P, P) is a product of the probability spaces 
(f2 n , P n , P n ) forn = 1, 2, .... Let ^f n be a function on £l n . Let X n (uji,uj2, •••) = 
* n (o; n ). Let Y n (uii,U2,—) = (5n)*K) and Z n (ui, u 2 , ...) = ^* n {u n ). Then 
P-almost surely we have (X n )* = Y n and X* = Z n . 

In particular, if X±, ...,X n are iidpnmrvs, then (Xi)*, (X n )* are identi- 
cally distributed independent random variables, and so are X^, ...,X*. Let 
S n = Xi + ■ ■ ■ + X n . From the Strong Law of Large Numbers as applied 
to {(X„)*} and {X*} (and using the fact that if \X±\* is integrable, then so 
are and X±) it then follows that almost surely: 

(1) EUX^J < liminf — < limsup— < Bfljl. 

n->oo n n->oo n 

Here and elsewhere "almost surely" will mean except perhaps on a set of 
probability zero. Thus an event holds almost surely provided its lower prob- 
ability is 1. (In the case of complete measures, this is equivalent to the usual 
notion of holding almost surely as holding on a set of full measure.) 
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We can define the lower and upper expectations of X\ as = -E[(Xl)*] 

and = £7LY*], respectively (for more on lower and upper expecta- 

tions, see [2]). Again, we have a trivial case when E^Xi)*] = and 
then the Strong Law holds. In that case, (X n )* = X* almost surely (since 
{X n )* < X n < X*), and X n will be measurable with respect to the comple- 
tion of P. 

We can now ask whether ([I]) can be improved on in any way. For instance, 
can the first or last almost sure inequality sometimes be replaced by an 
equality? Or can we say that in the non-trivial case it is almost surely 
true that S n /n diverges? Our main result shows that the answers to these 
questions are negative. 

Theorem 1. Let Xi,X%, ... be iidpnmrvs with E*[\Xi\] < oo. Suppose A is 
a non-empty proper subset of [E*[Xi], E*[Xi]]. Then each of the following 
is maximally nonmeasurable: 

(i) the subset of Q where lim inf n _>.oo S n (n)/n is in A 

(ii) the subset of Q where lim sup n ^. oc S n (n)/n is in A 

(hi) the subset of Q, where limj^oo S n (n) jn exists and is in A 

(iv) the subset of £1 where lim n _^ 00 S n (n)/n exists 

(v) the subset of Q where all the limit points of S n (n)/n are in A. 

Thus in the non-trivial case ([£7* [X\], E* [X\]] has a non-empty proper 
subset if and only if £^LYi] < E^LYi]) nothing can be probabilistically said, 
with respect to P, about the limit points of S n (n)/n except that all the limit 
points lie within [E^X^, E^Xx}}. 

For completeness, here is a somewhat analogous result about the Weak 
Law: 

Theorem 2. Let X\,Xi,... be iidpnmrvs with E*[\Xi\] < oo. Suppose a £ 
and that e > is sufficiently small that [E*[Xi],E*[Xi]] is 
notasubsetof[a—e,a+e]. Then P*(\S n /n—a\ > e) — > and P*{\S n /n—a\ > 
e) — > 1 as n — > oo. 

The proof of both theorems will be based on the following fact about the 
existence of extensions of measures. 

Lemma 1. Suppose f is a function on a probability space (Q, J 7 , P) and f 

is simple, i.e., takes on only finitely many values. Then there are extensions 

P* and P* of P defined on the a -field generated by J- and f such that f = f* 

almost surely with respect to P* and f = f* almost surely with respect to 
p* 

Remark. Our proofs of the theorems would be much simpler if we could have 
this for / taking on infinitely many values, but alas the lemma is false in 
that case. To see this falsity, let Q be the open square (0, l) 2 , make T be the 
cr-held of subsets of the form A x (0, 1) for A C (0, 1) Lebesgue-measurable, 
take P to be the restriction of Lebesgue measure to P, and set f(x,y) = y. 
Then /* = almost surely with respect to P but there is no extension of P 
with respect to which / = almost surely, since / is nowhere equal to zero. 
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2. Proofs 

The following simple fact will be useful. As usual, U A V = (U — V) U 
(V-U). 

Lemma 2. Suppose B is such that P*{B) = 0. If U and V are measurable 
sets such that U — B = V — B , then U A V is a null set. 

Proof. If U-B = V-B, then U-V C B and V-U C B. But if P* (5) = 0, 
then all measurable subsets of B are null sets. □ 

Proof of Lemma Q3 We shall prove the result about /*; the rest of the lemma 
follows by applying this to — /. 

We need only prove our result in the special case where / > everywhere 
and /* = almost surely. For suppose we have proved this special case and 
are given a general simple /. It is easy to check that if / is simple, we can 
choose /* to be simple, e.g., by setting = max{y : ui € {a/ : /(a/) > 

y}*}. Using the easy fact that for any measurable g we have (/ — <?)* = f* — g 
almost surely, we have (/ — /*)* = almost surely. Let h = f — /*. Then 
h > everywhere and h* = almost surely. By the special case of the 
lemma, there is an extension P* of P to the cr-field generated by T and h 
such that Pfc-almost surely h = 0. Thus P„-almost surely /* = g. But the 
cr-field generated by T and h is the same as that generated by J- and / 
since / and h differ only by a measurable function, and so our proof will be 
complete. 

So, suppose that / > everywhere and /* = almost surely. Let A = 
{oj : f(oj) > 0}. Observe that P*(A) = 0. For suppose that B C A is 
measurable. Let y\ be the smallest non-zero value that / takes. Then 
/ > Vi ■ Lb everywhere, and so /* > y\ ■ 1b almost surely, which implies 
that P{B) = 0. Thus the only measurable subsets of A are null, and so 
P«(A) = 0. 

Now, let T\ be the collection of all sets of the form (U n A) U (V n A c ) , for 
U and V in T . It is easy to check that this is equal to the cr-field generated 
by T and A\ . 

Define P X {{U n A) U (V D A c )) = P\{V). First we need to check that this 
is well-defined. Suppose (U D A) U (V D A c ) = (tT ; flA)U (V' n A c ). Then 
y-i = 7n J 4 c = 7'ni c = 7'-A Since P*(A) = 0, it follows from 
Lemma E] that V AV' is a null set. Thus P(V) = P(V') and so P x is well 
defined. 

It is an easy exercise to verify that P\ is a probability measure that 
extends P and satisfies P\ (A) = 0. Thus / = except on a Pi-null set. Let 
(f2, P*,J~i) be the completion of (f2, Pi, .Pi). Any function which is constant 
except on a set of measure zero is measurable with respect to the completion 
of the measure, and hence / is measurable with respect to J-^. Letting P* 
be the restriction of P* to the cr-field generated by T and /, the proof is 
complete. □ 
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For the proofs of the theorems we will need this simple consequence of [5j 
Lemma 1.2.2]. 

Lemma 3. If \f — g\ < e everywhere, then \ f* — g*\ < e and |/* — g*\ < e 
almost surely. 

Lemma 4. Suppose that f = g on a measurable set B. Then = g* and 
f* = g* almost surely on B. 

Proof. As usual, we only need to show half of the result, say that /* = 
g* almost surely on B. Let h(u) = f*(uj) for uj £ B and let h(oj) = 
min(f*(uj),g*(uj)) otherwise. Then h is a measurable function such that 
h > f. Hence h > f* almost surely. Then g* > h > f* almost surely on B. 
In the same way, we see that /* > g* almost surely on5. □ 

The following trivial lemma provides the strategy for the proof of our 
theorems. 

Lemma 5. Suppose that B is subset of a probability space (Q,J-,P) such 
that there are extensions P\ and P2 of P so that B is P\- and P2 -measurable 
with Pi{B) = X! and P 2 (B) = x 2 . Then P*{B) < x x and x 2 < P*{B). In 
particular, if P\{B) = and P 2 (B) = 1, then B is maximally nonmeasur- 
able. 

Proof. We have P,(B) = P(B m ) = Pi(B*) < P^B) = x x and P*(B) = 
P(B*) = P 2 (B*) > P 2 (B) = x 2 , where B* and B* are defined with respect 
to P. □ 

By Lemma [H we need to show that for each of the subsets of f2 mentioned 
in Theorem [TJ there is an extension of P that assigns measure zero to the 
subset and another that assigns it measure one. 

As we will soon see, the following lemma will yield all the results of 
Theorem [TJ 

Lemma 6. Suppose X\,X 2 , ... is a sequence of iidpnmrvs such that E[\X\\*\ < 
00. Then for any a € [£7*[Xi],.E'*[.X'i]] there is an extension P' of P such 
that P' -almost surely S n /n converges to a. If E^[X\] < E*[Xi] then there 
is an extension P" of P such that P" -almost surely S n diverges. 

Proof. Write Ep[f] for the expectation of / with respect to P, i.e., J n f(ui) dP{uj). 
The variables Xi,X 2 ,... are defined by X n (u>i, ui 2 , ...) = ^\{uj n ) on our prod- 
uct space f2 for some real- valued function ^1. 
Let X' n = X n ■ l{|x n |<n}- 

Let A Uj ± = {±X n > n}. Let A n = {\X n \ > n} = A n ^ + U A n -. Observe 
that A Uj + C {X* n > n} and A n ^ C {(X n )* < -n}. Let B n ' = {\X*\ > 
n}U {\(X n )*\ > n}. Clearly A n C B n . Then: 

CO OO OO 

(2) J2 P ( B ^ < E P (K X ™)*I > n) + Y, P (\K\ > n) < 00, 

n=l n=l n=l 
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since £~ =1 P(\X\ > n) < / °° P(\X\ >t)dt = E[\X\], and since both (X n ), 
and X* have finite expectations given that \X n \* does. In particular, almost 
surely, only finitely many of the B n occur by Borel-Cantelli. Since A n C B n , 
almost surely only finitely many of the A n occur. 

I now claim that E[(X' n )*} £7[(Xl)*] and E[(X' n )*} -> E[X*] as n ->• oo. 
We only need to prove the latter claim since the former follows by applying 
the latter to the iidpnmrv sequence {— X n }. Now, outside of B n , we have 
X' n = X n , and since B n is measurable it follows from Lemma S] that outside 
of B n we have {X' n )* = X* almost surely. Moreover, everywhere on B n we 
have X' n = and hence {X' n )* = by Lemma HI Thus: 



\E[(X' n y\ - E[X{] 



< 
< 

< 



\E[(X' n )*} - E[X*}\ 

E[\(x' n y - x* n \] 
E[\(x' n y -x:\-i Bn } 

E[\X* n \-l B J 

E [\ X n\ • l {\X n \*>n}] 
E [\ X *\ ■ 1 {|X 1 |*>n}] -> 



0, 



since P[|Xi|*] < oo. 

Let = ^ ■ l{|^|< n } so that X' n (uji,u}2, —) = W n (u) n ). Choose a simple 
function \Pi jn such that both |\&i )n — ^f'\ < 1/n and l^inl — n everywhere 
on Hi. 

By Lemma Q] there is an extension Pi n o °f Pi such that is Pi jn ,o- 
measurable and Pi )n> o-almost surely \Pi jn = (^i >n )*, and an extension Pi, n ,i 
of Pi such that ^\ )n ,n is Pi jnj i-measurable and Pi jnj i-almost surely ^>\ in = 
^\ n . Let T\^ n be the cr-field on S7i generated by Pi and ^i^. 

For i = 1,2, let P % be the product of the measures Pi,i,i, P\,2,i, Pi,3,i, ■■■■ 
This is an extension of P. 

Let Y n (ui,uj2,...) = fi, n K). By Proposition [H (Y n )*(u)i, w 2 , ••■) = 
(^ , i,n)*(w„) for P-almost all (u±, 002, ■■■)■ Then P°-almost surely we have 
Y n = (y n )* and P 1 -almost surely Y n = Y* (where (Y n )* and Y* are de- 
fined with respect to P). Define the measure P' = (1 — a)P° + aP 1 , where 
a is such that a = (1 - o)£7j>[(Xi)»] + a£p[XJ]. Then £p/[F n ] = (1 - 
a)E P [(Y n )*} + aEp[Y*}. Now |(F n )* - ( x 'n)*\ < V" and |Y n * - {X' n )*\ < 1/n 
everywhere by Lemma [3] and the choice of ^^n, so by uniform convergence 
we have P[(y n )* — (X' n )*\ and E[Y* — (X' n )*] converging to zero. Moreover, 
we have already seen that E[(X' n )*] -> E[(Xi\] and E[(X' n )*} -> E[X£], 
so E P [{Y n y] -> E P [(Xi)*] and E P [YJ] -> £? P [Xf]. Thus, £7 P ,[y n ] -> 
(1 - a)P P [(Xi),] + aEpft*] = a. 
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Now \Y n — X' n \ < 1/n everywhere and \X' n \ < max(|(X n )*|, • lp n . 
Thus: 



~ VarpTO < " E[Y*} 

n=l n=l 



Ep-[((X; + (Y„ - X' n )f\ 



n 2 



< 



< 



n=l 

n _f, E P ,[(X^] | ^ 2 

n=l ra=l 

^ gp-[W.f'i«j , 2 y E P ,[(x:f.i Bn \ | ~ 2 

■^-^ n 2 ^ n 2 n 3 

n=l n=l n=l 

n=l n=l 
2 Ep[((X n )*) 2 ■ l{|(x n ).|>n}] | 2 ^[(^n) 2 ' 1 {|Ag|>n}] 
n=l n=l 



+0(1), 



where the last equality follows from the fact that P is an extension of P' and 
(Xn)* and X* are P-measurable. The finiteness of the right hand side then 
follows from the proof of [U Lemma 2.4.3] . It then follows from Kolmogorov's 

Strong Law gj Theorem 5.8] that P'-almost surely (Y% -\ h Y n - E[Yi + 

1- 5^]) -> and hence (Y 1 -\ + F„)/n ->• a. 

Let 5 = Ur=i B n- Let T n = Y x + • • • + Y„. Let S' n = X[ + ■ ■ ■ + X' n . Since 
\X' n -Y n \ < 1/n everywhere, \S' n /n-T n /n\ < (l/n)(l + 1/2 + • • • + 1/n) and 
the right hand side converges to zero. Moreover, P-almost surely for all but 
finitely many n we have X n = X' n , so that P-almost surely (S n — S' n )/n — > 0. 
Thus, P-almost surely we have (S n — T n )/n — > 0. But P' extends P, so this 
also holds P'-almost surely. But since P'-almost surely T n /n — > a, we also 
have S n /n converging P'-almost surely to a. 

To construct P", first recall that E P [(Y n )*\ -> E P [(Xi)„] and E P [Y*] -> 
E P [X{\. Let 7 = Ep[(X\)*\ and 5 = Ppps^*]. For n > 0, let a n be a 
strictly increasing sequence of positive integers such that (a) |Pp [(!&)*] — 
Ep[(X x )*\\ < 1/n and \E P [Y*] - E P [X{}\ < 1/n for all k > a n , and 
(b) a n+ \/a n — > 00. Let ao = 1. 

Let L n = {a n _i + 1, a n _i + 1, a n }. Let L = L\ UL3UL5U .... For n € L, 

let a n = Pp[(y n )*] and for n ^ L, let a n = _Ep[Y^]. Let /3 n = a\ H h a n . 

I now claim that /3 a2n /a 2n -> 5 and /3 a2 „+i /«2n+i -f 7 as rn> 00. For, 
since both Pp[(y n )^] and £p[Y^] converge, there is an M < 00 such that 
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| CKfe | < M for all K, and then by the choice of a n : 

Cl2n 02n-l 

Pa 2n - a 2n 5 = ^2 (Ep[Y£] -5) - a 2n -iS + ^ a k 

k=a,2„-i+l k=l 

= 0(a 2n /(2n - 1)) + 0(a 2 n-i). 

Since a2n-i/a2n — >■ by condition (b) above, we see that (3 a2n /a 2n — <5 
converges to as desired. Likewise: 

a 2n + l <J2n 

/3a 2 „+i - 02n+l7 = Yl ( E P[( Y k)*]-l) ~ 0-2nl + ^ak 

= 0(a 2n+1 /2n) + 0(a 2n ), 

and so /3 a2n+1 / 'a 2n +i -> 7- 

Now define P[' n to be equal to Pi, n ,o if n € L, and to Pi,n,i n ^ L. Let 
P" be the product of the measures P{\, P{' 2 , •••• 

In exactly the same way as we proved above using Kolmogorov's Strong 
Law that P'-almost surely (T n — Epr[T n ]), we can also show that P"-almost 
surely (T n - Ep»[T n ])/n -4 0. But E P ,i[T n ] = (3 n since E P »[Y n ] = Ep[(Y n )*\ 
if n 6 L and Epn [Y n ] = E P [Y*] otherwise. Thus, P"-almost surely T a2n /a 2n — > 
5 and T a2n+1 /a 2n+1 -)■ 7. 

But we have already seen that (S n /n — T n )/n converges to zero P-almost 
surely, and hence also P"-almost surely. Thus P"-almost surely S a2n / a 2n — > 
5 and S a2n+1 /a 2n+ i — > 7. Since 5 > 7, our desired divergence result follows. 

□ 

Proof of Theorem [H Choose any a\ £ A By Lemma (HJ we have an exten- 
sion P' of P such that P'-almost surely S n /n converges to a±. Moreover, 
since A is a proper non-empty subset of [Pp[(Xl)*], Pp[X^]], the latter in- 
terval must contain at least two points and hence Ep[(Xi)*] < Ep[X*], so 
by the same lemma there is an extension P" of P such that P"-almost surely 
S n /n diverges. Now choose any a 2 € [.Ep[(Xl)*], E'pfX*]] — A. Again, by 
the same lemma there is an extension P'" of P such that P"'-almost surely 
S n /n converges to a 2 . 

All the events described in (i)-(v) will happen whenever S n /n — > ai, and 
so they all have P'-measure 1. Events (i), (ii), (iii) and (v) cannot happen 
when S n /n — > a 2 , and so they all have P'"-measure 0. And event (iv) as 
P"-measure 0. Thus, each event has measure 1 under one extension of P 
and measure under some other extension, and so by Lemma each event 
is maximally P-nonmeasurable. □ 

Proof of Theorem [21 Let a and e be as in the statement of the theorem. The 
conditions of the theorem guarantee that there is a point b £ [P* [Xi], E* [Xi]] — 
[a — e, a + e\. Let 7 = \b — a\ — e. Since b £ [a — e, a + e], we have 7 > 0. 

By Lemma[6l let P' be an extension of P such that P'-almost surely S n /n 
converges to a, and let P" be an extension of P such that P"-almost surely 
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S n /n converges to b. Then S n /n converges to a in P'-probability and to b 
in P"-probability. Hence lim n _>oo P'i\S n /n — a| > e) = 0. By LemmaEl we 
have linin^oo P^Sn/n— a\ > e) = 0. Moreover, = linim^oo P" (\S n /n— b\ > 
7) > limsup n _ >00 P"( I S n /n— a \ < e) by choice of 7. Thus P"(\S n /n— a\ < e) 
converges to 0, and so P"(\S n /n — a\ > e) converges to 1, so that P*(\S n /n — 
a\ > e) also converges to 1. □ 
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